A multi-block alternating direction method with parallel splitting for decentralized consensus optimization

Decentralized optimization has attracted much research interest for resource-limited networked multi-agent systems in recent years. Decentralized consensus optimization, which is one of the decentralized optimization problems of great practical importance, minimizes an objective function that is the sum of the terms from individual agents over a set of variables on which all the agents should reach a consensus. This problem can be reformulated into an equivalent model with two blocks of variables, which can then be solved by the alternating direction method (ADM) with only communications between neighbor nodes. Motivated by a recently emerged class of so-calledmulti-block ADMs, this article demonstrates that it is more natural to reformulate a decentralized consensus optimization problem to one with multiple blocks of variables and solve it by a multi-block ADM. In particular, we focus on the multi-block ADM with parallel splitting, which has easy decentralized implementation. Convergence rate is analyzed in the setting of average consensus, and the relation between two-block and multi-block ADMs are studied. Numerical experiments demonstrate the effectiveness of the multi-block ADM with parallel splitting in terms of speed and communication cost and show that it has better network scalability. Introduction In recent years, the communication, signal processing, control, and optimization communities have witnessed considerable research efforts on decentralized optimization for networked multi-agent systems [1-3]. A networked multi-agent system, such as a wireless sensor network (WSN) or a networked control system (NCS), is composed of multiple geographically distributed but interconnected agents which have sensing, computation, communication, and actuating abilities. This system generally has limited resources for communication, since battery power is limited and recharging is difficult, while communication between two agents is energy-consuming. Furthermore, the communication link is often vulnerable and bandwidth-limited. In this situation, decentralized optimization emerges as an effective approach to improve network scalability. In decentralized optimization, data and computation are decentralized. Each agent exchanges *Correspondence: [email protected] 1Department of Automation, University of Science and Technology of China, Hefei, Anhui, China Full list of author information is available at the end of the article information with its neighbors and accomplishes an otherwise centralized optimization task. This article focuses on the decentralized consensus optimization problem. We consider a network of L agents which cooperatively optimize a separable objective function [3-8]: